Vector Equations

Many parts of this page are based on Linear Algebra and its Applications, by David C. Lay

Vectors

We’re going to expand our view of what a linear system can represent.

Instead of thinking of it as a collection of equations, we are going to think about it as a single equation.

This is a major shift in perspective that will open up an entirely new way of thinking about matrices.

(…and it’s going to open the door to computer graphics, machine learning, and statistics … later on!)

To make this fundamental shift we need to introduce the idea of a vector. We’ll mainly talk about vectors today.

We’ll then return to thinking about a linear system – now interpreted as a vector equation – in the next lecture.

A matrix with only one column is called a column vector, or simply a vector.

Here are some examples:

\[ \begin{array}{ccc} {\bf u} = \left[\begin{array}{r}3\\-1\end{array}\right] & {\bf v} = \left[\begin{array}{c}.2\\.3\end{array}\right] & {\bf w} = \left[\begin{array}{c}w_1\\w_2\end{array}\right] \end{array} \]

We will always use boldface for vectors – so you can tell that \(\mathbf{u}\) is a vector, while \(u\) is just an ordinary number.

We will use the term scalar to refer to ordinary numbers (not vectors).

The numbers within the vector are called components.

Because each of these vectors has two components, we say these vectors are in \(\mathbb{R}^2.\)

There is an important notational point here:

When we write \(\mathbb{R}^n\), we mean all the vectors that have exactly \(n\) components.

Here are some vectors in \(\mathbb{R}^3\):

\[ \begin{array}{ccc} {\bf u} = \left[\begin{array}{c}2\\3\\4\end{array}\right] & {\bf v} = \left[\begin{array}{r}-1\\0\\2\end{array}\right] & {\bf w} = \left[\begin{array}{c}w_1\\w_2\\w_3\end{array}\right] \end{array} \]

Vectors are Fundamental Objects

We are going to define operations over vectors, so that we can write equations in terms of vectors.

In particular, we now define how to compare vectors, add vectors, and multiply vectors by a scalar.

First: Two vectors are equal if and only if their corresponding entries are equal.

Thus \(\left[\begin{array}{c}7\\4\end{array}\right]\) and \(\left[\begin{array}{c}4\\7\end{array}\right]\) are not equal.

Next: Multiplying a vector by a scalar is accomplished by multiplying each entry by the scalar.

For example:

\[3 \left[\begin{array}{r}1\\-2\end{array}\right] = \left[\begin{array}{r}3\\-6\end{array}\right]\]

And finally: the sum of two vectors is the vector whose entries are the corresponding sums.

For example:

\[\left[\begin{array}{r}1\\-2\end{array}\right] + \left[\begin{array}{c}2\\5\end{array}\right] = \left[\begin{array}{r}1 + 2\\-2 + 5\end{array}\right] = \left[\begin{array}{c}3\\3\end{array}\right].\]

Note that vectors of different sizes cannot be compared or added.

For example, if \({\bf u} \in \mathbb{R}^2\) and \({\bf v} \in \mathbb{R}^3\):

• we cannot ask whether \({\bf u} = {\bf v}\), and
• \({\bf u} + {\bf v}\) is undefined.

So with these three definitions, we have all the tools to write equations using vectors.

For example, if \({\bf u} = \left[\begin{array}{r}1\\-2\end{array}\right]\) and \({\bf v} = \left[\begin{array}{r}2\\-5\end{array}\right]\) then

\[ 4{\bf u} - 3{\bf v} = \left[\begin{array}{r}-2\\7\end{array}\right]\]

Vectors Correspond to Points

As already noted, an ordered sequence of \(n\) numbers can be thought of as a point in \(n\)-dimensional space.

Hence, a vector like \(\left[\begin{array}{c}-2\\-1\end{array}\right]\) can be thought of as a point on the plane.

We will sometimes write vectors as a list, like this: \((-2,-1).\)

(which lists the vector components from top to bottom).

Sometimes we draw an arrow from the origin to the point.

This style comes from physics, but can be a helpful visualization in any case.

Vector Addition, Geometrically

A geometric interpretation of vector sum is as a parallelogram.

If \({\bf u}\) and \({\bf v}\) in \(\mathbb{R}^2\) are represented as points in the plane, then \({\bf u} + {\bf v}\) corresponds to the fourth vertex of the parallelogram whose other vertices are \({\bf u}, 0,\) and \({\bf v}\).

This should be clear from the definition of vector addition (i.e., addition of corresponding elements).

Vector Scaling, Geometrically

For a given vector \({\bf v}\) and a scalar \(a\), multiplying \(a\) and \({\bf v}\) corresponds to lengthening \(\bf v\) by a factor of \(a\).

So \(2\bf v\) is twice as long as \(\bf v\).

Multiplying by a negative value reverses the “direction” of \(\bf v\).

Once Loop Reflect

The Algebra of \(\mathbb{R}^n\)

We’ve defined a new object (the vector), and it has certain algebraic properties.

1. \({\bf u} + {\bf v} = {\bf v} + {\bf u}\)
2. \(({\bf u} + {\bf v}) + {\bf w} = {\bf u} + ({\bf v} + {\bf w})\)
3. \({\bf u} + {\bf 0} = {\bf 0} + {\bf u} = {\bf u}\)
4. \({\bf u} + ({\bf -u}) = {\bf -u} + {\bf u} = {\bf 0}\)
5. \(c({\bf u} + {\bf v}) = c{\bf u} + c{\bf v}\)
6. \((c+d){\bf u} = c{\bf u} + d{\bf u}\)
7. \(c(d{\bf u}) = (cd){\bf u}\)
8. \(1{\bf u} = {\bf u}\)

You can verify each of these by from the definitions of vector addition and scalar-vector multiplication.

Linear Combinations

A fundamental thing we will do is to construct linear combinations of vectors:

\[ {\bf y} = c_1{\bf v_1} + ... + c_p{\bf v_p} \]

The \(c_i\) values are called weights. Weights can be any real number, including zero. So some examples of linear combinations of \(\mathbf{v_1}\) and \(\mathbf{v_2}\) are:

\[ \sqrt{3}{\bf v_1} + {\bf v_2}, \]

\[ \frac{1}{2}{\bf v_1} \;\; (= \frac{1}{2}{\bf v_1} + 0{\bf v_2}) \] and

\[ {\bf 0} \;\;(= 0{\bf v_1} + 0{\bf v_2}) \]

Linear Combinations, Geometrically

\[{\bf w} = 2{\bf u} + {\bf v}\]

\[{\bf p} = -{\bf u} - {\bf v}\]

\[{\bf q} = -{\bf u} + 3{\bf v}\]

A Fundamental Question

We are now going to take up a very basic question that will lead us to a deeper understanding of linear systems.

Given some set of vectors \({\bf a_1, a_2, ..., a_k}\), can a given vector \(\bf b\) be written as a linear combination of \({\bf a_1, a_2, ..., a_k}\)?

Let’s take a specific example.

Let \({\bf a_1} = \left[\begin{array}{c}1\\-2\\-5\end{array}\right], {\bf a_2} = \left[\begin{array}{c}2\\5\\6\end{array}\right],\) and \({\bf b} = \left[\begin{array}{c}7\\4\\-3\end{array}\right]\).

We need to determine whether \({\bf b}\) can be generated as a linear combination of \({\bf a_1}\) and \({\bf a_2}\). That is, we seek to find whether weights \(x_1\) and \(x_2\) exist such that

\[ x_1{\bf a_1} + x_2{\bf a_2} = {\bf b}.\]

Once Loop Reflect

Solution. We are going to convert from a single vector equation to a set of linear equations, that is, a linear system.

We start by writing the form of the solution, if it exists:

\[ x_1{\bf a_1} + x_2{\bf a_2} = {\bf b}.\]

Written out, this is:

\[ x_1 \left[\begin{array}{c}1\\-2\\-5\end{array}\right]+ x_2\left[\begin{array}{c}2\\5\\6\end{array}\right] = \left[\begin{array}{c}7\\4\\-3\end{array}\right].\]

By the definition of scalar-vector multiplication, this is:

\[\left[\begin{array}{c}x_1\\-2x_1\\-5x_1\end{array}\right]+ \left[\begin{array}{c}2x_2\\5x_2\\6x_2\end{array}\right]= \left[\begin{array}{c}7\\4\\-3\end{array}\right].\]

By the definition of vector addition, this is:

\[\left[\begin{array}{r}x_1 + 2x_2\\-2x_1 + 5x_2\\-5x_1+6x_2\end{array}\right] = \left[\begin{array}{c}7\\4\\-3\end{array}\right].\]

By the definition of vector equality, this is:

\[\begin{array}{rcl}x_1 + 2x_2&=&7\\-2x_1 + 5x_2&=&4\\-5x_1+6x_2&=&-3\end{array}.\]

We know how to solve this! Firing up Gaussian Elimination, we first construct the augmented matrix of this system, and then find its reduced row echelon form:

\[\left[\begin{array}{rrr}1&2&7\\-2&5&4\\-5&6&-3\end{array}\right] \sim \left[\begin{array}{rrr}1&2&7\\0&9&18\\0&16&32\end{array}\right] \sim \left[\begin{array}{rrr}1&2&7\\0&1&2\\0&16&32\end{array}\right] \sim \left[\begin{array}{rrr}1&0&3\\0&1&2\\0&0&0\end{array}\right]\]

Voila!

Now, reading off the answer, we have \(x_1 = 3\), \(x_2 = 2\). So we have found the solution to our original problem:

\[ 3 \left[\begin{array}{c}1\\-2\\-5\end{array}\right]+ 2\left[\begin{array}{c}2\\5\\6\end{array}\right] = \left[\begin{array}{c}7\\4\\-3\end{array}\right].\]

In other words, we have found that

\[ 3{\bf a_1} + 2{\bf a_2} = {\bf b}.\]

Once Loop Reflect

A Vector Equation is a Linear System!

And vice versa!

Let’s state this formally. First, of all, recalling that vectors are columns, we can write the augmented matrix for the linear system in a very simple way.

For the vector equation

\[x_1{\bf a_1} + x_2{\bf a_2} = {\bf b},\]

the corresponding linear system has augmented matrix:

\[[{\bf a_1}\;{\bf a_2}\;{\bf b}].\]

Then we can make the following statement:

A vector equation

\[ x_1{\bf a_1} + x_2{\bf a_2} + ... + x_n{\bf a_n} = {\bf b} \]

has the same solution set as the linear system whose augmented matrix is

\[ [{\bf a_1} \; {\bf a_2} \; ... \; {\bf a_n} \; {\bf b}].\]

This is a powerful concept; we have related

• a single equation involving columns
• to a set of equations corresponding to rows.

Span

If a vector equation is equivalent to a linear system, then it must be possible for a vector equation to be inconsistent as well.

How can we understand what it means – in terms of vectors – for a vector equation to be inconsistent?

The answer involves a new concept: the span of a set of vectors.

Let’s say we are given vectors \({\bf a_1}, {\bf a_2},\) and \({\bf b}\).

And, say we know that it is possible to express \(\mathbf{b}\) as a linear combination of \(\mathbf{a}_1\) and \(\mathbf{a}_2\).

That is, there are some \(x_1, x_2\) such that \(x_1{\bf a_1} + x_2{\bf a_2} = {\bf b}.\)

Then we say that \({\bf b}\) is in the Span of the set of vectors \(\{{\bf a_1}, {\bf a_2}\}.\)

More generally, let’s say we are given a set of vectors \({\bf v_1, ..., v_p}\) where each \({\bf v_i} \in \mathbb{R}^n.\)

Then the set of all linear combinations of \({\bf v_1, ..., v_p}\) is denoted by

\[ \operatorname{Span}\{{\bf v_1, ..., v_p}\}\]

and is called the subset of \(\mathbb{R}^n\) spanned by \({\bf v_1, ..., v_p}.\)

Span of a single vector in \(\mathbb{R}^3\)

Once Loop Reflect

Span of two vectors in \(\mathbb{R}^3\)

Once Loop Reflect

Asking Whether a Vector Lies Within a Span

Asking whether a vector \({\bf b}\) is in Span\(\{{\bf v_1, ..., v_p}\}\) is the same as asking whether the vector equation

\[x_1{\bf v_1} + x_2{\bf v_2} + \dots + x_p{\bf v_p} = {\bf b}\]

has a solution.

… which we now know is the same as asking whether the linear system with augmented matrix

\[ [{\bf v_1} \; {\bf v_2} \; ... \; {\bf v_p} \; {\bf b}]\] has a solution.

Let \({\bf a_1} = \left[\begin{array}{c}1\\-2\\3\end{array}\right], {\bf a_2} = \left[\begin{array}{c}5\\-13\\-3\end{array}\right],\) and \({\bf b} = \left[\begin{array}{c}6\\8\\-5\end{array}\right]\).

Then Span{\(\bf a_1, a_2\)} is a plane through the origin in \(\mathbb{R}^3\). Is \(\bf b\) in that plane?

Solution: Does the equation \(x_1{\bf a_1} + x_2{\bf a_2} = \bf b\) have a solution?

To answer this, consider the equivalent linear system. Solve the system by row reducing the augmented matrix

\[[{\bf a_1} \; {\bf a_2} \; {\bf b}]:\]

\[\left[\begin{array}{rrr}1&5&6\\-2&-13&8\\3&-3&-5\end{array}\right] \sim \left[\begin{array}{rrr}1&5&6\\0&-3&20\\0&-18&-23\end{array}\right] \sim \left[\begin{array}{rrr}1&5&6\\0&-3&20\\0&0&-143\end{array}\right]\]

The third row shows that the system has no solution.

This means that the vector equation \(x_1{\bf a_1} + x_2{\bf a_2} = \bf b\) has no solution.

So \({\bf b}\) is not in Span{\(\bf a_1, a_2\)}.

What does this situation look like geometrically?

Once Loop Reflect

Two Distinct Vector Spaces

Be sure to keep clear in your mind that we have been working with two different vector spaces.

One vector space is for visualizing equations that correspond to rows.

The other vector space is for visualizing vector equations (involving columns).

These are two different ways of visualizing the same linear system.

Let’s look at an inconsistent system both ways:

Here are the views, first as a vector equation, and then as a system of equations.

Once Loop Reflect

Once Loop Reflect

The second vector space (and visualization) is more familiar to us right now.

However, as the course goes on we will use the first vector space – the one in which columns are vectors – much more often. Make sure you understand the figure!